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The  Application  of  Statistical 
Decision  Theory  to  a  Missile  Testing  Problem^ 

ABSTRACT 

Experiments  are  planned,  performed,  and  analyzed  for  two  main 
pvirposes:  first,  to  discover  facts;  and  second,  to  use  those  facts 
in  making  decisions. 

Classical  statistics  typically  considers  these  two  purposes 
separately,  with  special  emphasis  on  discovering  (or  estimating)  the 
facts . 

Statistical  decision  theory,  on  the  other  hand,  considers  each 
problem  as  a  unified  whole,  so  that  the  experiment  and  its  analysis 
are  greatly  Influenced  (a)  by  the  decision  which  is  to  be  made,  and 
(b)  by  the  cost  of  the  experiment  itself. 

As  the  price  for  treating  more  of  the  important  factors  explicit¬ 
ly,  statistical  decision  theory  requires  more  information  than  the 
classical  methods.  On  the  other  hand,  problems  can  be  treated  which 
are  inaccessible  to  the  classical  methods,  for  example;  "Shall  we 
do  any  testing  at  all?" 


♦This  paper  is  a  revised  and  expanded  version  of  a  paper  delivered 
at  the  World-Wide  Operations  Analysis  Technical  Conference  -  12-13 
October  I96I 
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1.  Introduction 


In  this  paper,  we  wish  to  describe  two  approaches,  each  applicable 
to  a  wide  variety  of  statistical  problems.  For  expository  purposes, 
we  have  chosen  to  apply  both  these  approaches  —  which  we  call,  respec¬ 
tively,  the  classical  approach  and  the  statistical  decision  theory 
approach  --  to  a  specific  problem.  In  order  to  clarify  the  principles 
which  we  are  presenting,  we  have  sacrificed  the  realism  of  the  example 
by  making  several  simplifying  assumptions.*  Of  course,  both  the 
methods  discussed  could  be  applied  without  making  these  simplifying 
assumptions.  Just  as  they  could  be  applied  to  a  wide  variety  of  other 
real  problems. 

The  central  ideas  of  each  of  these  approaches  can  be  distinguished 
as  follows: 

a.  Classical  Statistics: 

"What  are  the  facts?" 

b.  Statistical  Decision  Theory: 

"What  shall  we  do?" 

Because  of  the  difference  In  these  central  Ideas,  there  Is  a  great 
difference  between  the  two  approaches  In  the  parts  of  the  total  pro¬ 
blem  which  are  included  In  the  mathematical  model.  Statistical 
decision  theory,  which  focuses  attention  on  the  pragmatic  question 
"What  shall  we  do?",  takes  Into  consideration  factors  which  are  typi¬ 
cally  excluded  from  the  classical  approach. 


♦Two  specific  examples  of  this  simplification:  (a)  We  assume  that 
all  targets  have  the  same  value  to  the  attacker  and  that  all  are 
equally  hardj  (b)  we  treat  reliability  alone,  whereas  real  problems 
require  that  reliability  and  accuracy  be  treated  together  In  the 
same  model. 
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After  reviewing  both  approaches,  we  will  Illustrate  them  by  apply¬ 
ing  them  to  a  specific  example.  The  example  concerns  the  question, 

"How  many  missiles  should  be  expended  in  operational  test  firings?" 

It  is  Important  to  realize  that  we  are  not  concerned  with  R  and  D  test 
firings,  whose  primary  purpose  is  to  improve  missile  design  and  reli¬ 
ability,  but  only  with  the  later  firings  from  operationally  conflgxired 
sites.  The  primary  purposes  of  those  later  firings  are  (a)  to  evalu¬ 
ate  missile  reliability,  and  (b)  to  provide  guidance  for  the  allocation 
of  missiles  to  targets. 

2.  Classical  Statistics 

Classical  statistics  is  concerned  with  the  theory  of  how  to  use 
the  results  of  an  experiment  (a)  to  test  hypotheses,  or  (b)  to  esti¬ 
mate  parameters.  (A  hypothesis  might  be  any  statement  whose  truth  is 
unknown,  such  as  "Missile  reliability  is  greater  than  A  para¬ 

meter  might  be  any  variable  whose  true  value  is  unknown,  such  as  the 
reliability  of  our  missiles.) 

A  parameter  estimate  may  be  either  (l)  a  point  estimate,  or  (2) 
an  interval  estimate.  (A  point  estimate  is  a  statement  such  as  "Based 
on  these  test  results,  reliability  is  O.63."  An  interval  estimate  la 
a  statement  of  the  form  "Based  on  these  test  results,  we  can  say  with 
805^  confidence  that  the  reliability  is  between  0.53  and  O.7I.")  The 
latter  type  of  estimate  tells  us  how  trustworthy  the  specific  numbers 
may  be. 

a.  Theory  of  Estimation. 

Classical  statistics  emphasizes  the  development  of  theory 
to  guide  one  in  choosing  functions  of  the  sample  observations  for 
testing  hypotheses,  and  for  point  and  Interval  estimation.  Not 
always  Ignored,  but  usually  omitted  from  explicit  treatment  in  the 
mathematical  model,  are  considerations  of  testing  coats  and  the  loss 
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associated  with  making  a  vrrong  estimate.  There  Is  no  suggestion  In¬ 
tended  here  that  classical  statistics  could  not  take  these  things 
Into  account.  Implicitly  or  even  explicitly)  however.  It  Is  not 
usually  done.  For  example,  a  brief  search  through  several  standard 
statistics  texts  reveals  no  mention  at  all  of  the  cost  of  testing, 
except  by  Implication  In  the  discussion  of  sequential  testing  (since 
one  of  the  stated  advantages  of  sequential  testing  Is  that  it  de¬ 
creases  the  average  number  of  items  tested). 

b.  Design  of  Experiments 

We  should  also  remark  here  that  the  large  body  of  litera¬ 
ture  on  the  "design  of  experiments"  does  not  (in  general)  refer  ex¬ 
plicitly  to  the  cost  of  experimentation,  but  asks  questions  like  "How 
much  testing  Is  necessary  to  achieve  the  precision  and  confidence  re¬ 
quired?"  Of  course.  If  the  answer  to  this  question  Implies  an  un¬ 
reasonably  costly  experiment,  the  words  "necessary"  and  "required" 
will  be  redefined  and  the  question  will  be  asked  again.  In  this 
way  the  costs  ^  have  some  effect  on  the  classically -designed  experi¬ 
ment,  but  In  a  subjective  and  Imprecise  way.  In  fact,  most  of  the 
"design  of  experiments"  Is  directed  to  the  very  different  problem  of 
avoiding  undeslred  Interactions  between  the  things  being  measured  or 
tested  and  random  or  Irrelevant  disturbances. 

3.  Statistical  Decision  Theory 

Statistical  decision  theory,  on  the  other  hand,  uses  the  same 
mathematical  theory  of  probability  which  underlies  classical  statis¬ 
tics,  but  Increases  the  scope  of  the  mathematical  model  to  Include 
explicit  consideration  of  (a)  cost  of  testing,  (b)  decisions  which 
may  be  influenced  by  teat  results,  (c)  cost  of  making  the  wrong 
decision,  and  (d)  Judgments  on  the  likelihood  of  the  various  results 
(prior  probabilities).  One  penalty  for  Increasing  the  scope  of  the 
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model  is,  of  course.  Increased  complexity.  Another  fact,  often  re¬ 
garded  as  a  penalty,  is  that  statistical  decision  theory  gives  re¬ 
sults  which  depend  explicitly  on  the  above  four  factors,  so  that 
the  applicability  of  the  results  is  usually  limited  to  the  one  speci¬ 
fic  problem  considered.  However,  we  believe  that  those  factors  logi¬ 
cally  should  have  an  effect  on  the  amount  of  testing,  and  on  the 
actions  taken  after  observing  the  results,  so  that  the  generality 
obtained  by  other  methods  may  be  Illusory. 

The  question,  as  we  see  it.  Is  simply  whether  those  four 
factors  should  be  taken  Into  account  by  explicitly  including  them 
in  the  model,  or  whether  they  should  be  taken  into  account  by  rely¬ 
ing  on  the  subjective  Judgment  of  the  statistician.  In  the  case 
of  an  experienced  statistician,  whose  experience  Is  relevant  to  the 
problem  at  hand,  the  classical  approach  will  be  more  convenlentj 
however,  we  can  see  no  method  of  determlng  whether  the  statistician's 
experience  is  In  fact  relevant  except  by  comparing  his  opinions  with 
objective  calculations  which  take  the  additional  factors  into  account 
explicitly.  For  most  operations  analysis  problems,  and  for  our  exam¬ 
ple  in  particular,  the  additional  information  needed  for  statistical 
decision  theory  can  be  made  available  by  reasonable  assumptions. 

4.  Our  Specific  Missile  Testing  Problem 

The  missile  testing  problem  has  two  parts;  first,  "How  many 
missiles  should  be  fired  in  an  operational  test  program?"  and 
second,  "How  should  the  results  of  the  test  progrem  be  used  as  a 
guide  in  targeting?" 

a.  The  Classlcal-Statlatlcal  Approach 

The  question  of  how  many  missiles  to  fire  can  be  partially 
answered  by  the  methods  of  classical  statistics.  Before  explaining 
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the  method  used,  we  will  describe  how  confidence  limits  for  reli¬ 
ability  are  determined. 

(1)  Confidence  Limits 

The  confidence  limits  are  based  on  the  mathematical 
theory  of  probability  and  the  binomial  theorem,  and  are  summarized 
In  graphs  of  the  type  shown  In  Pig.  1.*  The  true  reliability,  R, 

Is  plotted  on  the  ordinate  and  the  observed  fraction  of  successes 
In  a  sample,  r^.  Is  plotted  on  the  abscissa.  Each  pair  of  curves 
gives  upper  confidence  limit  Ry  and  lower  confidence  limit  Rj^, 
for  a  specific  sample  size  (number  of  missiles  tested)  and  a  speci¬ 
fic  value  of  the  confidence  coefficient  C,  whose  significance  will 
appear  In  a  moment.  One  enters  the  appropriate  graph  with  the  ob¬ 
served  fraction  of  successes,  r^,  draws  a  vertical  line,  and  then 
reads  R^  from  the  upper  curve  and  from  the  lower.  Since  Ry  and 
Rj^  depend  on  r^,  the  observed  fraction  of  successes,  which  la  of 
course  a  random  variable,  they  too  are  random  variables.  The  curves 
are  so  calculated  that.  If  one  determines  Ry  and  R^^  In  this  way 
from  each  of  many  experiments,  using  the  same  confidence  coefficient 
C  each  time,  then  the  random  variables  Ry  and  R^  will  in  fact  bracket 
the  true  probability  R  a  fraction  C  of  the  time. 

(2)  Choice  of  Confidence-Coefficient 

The  experimenter  chooses  C.  Of  course,  he  wants  a 
large  Cj  that  Is,  he  wants  a  high  probability  of  covering  the  true 
value.  He  also  wants  a  narrow  band  for  R,  that  is  he  wants  Ry  -  Rj^ 
to  be  small.  But,  for  a  given  sample  size,  as  C  Increases,  Ry  -  Rj^ 
also  Increases,  (as  Illustrated  In  Figure  1)  so  the  experimenter 
must  compromise.  And  It  Is  here  that  the  first  weakness  of  classical 
statistical  theory  shows  up.  It  offers  no  guidance  as  to  how  to 
resolve  the  conflict  between  the  desire  for  large  C  and  the  desire 
for  small  Ry  -  Ry.  Published  aids  for  computing  Ry  and  Ry  are 

♦See  Page  32. 
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typically  for  C  »  .8,  .9,  .95  and  .99,  and  so  a  value  from  this 
set  Is  usually  chosen.  But  the  practical  Implications  of  the  choice 
of  C  and  of  Ry  -  Rj^  are  not  ordinarily  considered  In  the  classical 
theory,  and  are  very  difficult  even  to  state  correctly. 

(3)  Choice  of  Number  Tested 

Further,  If  both  large  C  and  small  R^  -  Rj^  are  required, 
n  must  be  large.  Here  again,  the  classical  theory  provides  no 
guidance  on  how  to  resolve  the  conflict  between  the  desire  for  large 
C  and  small  R^  -  Rj^  on  the  one  hand,  and  the  cost  of  large  n  on  the 
other.  Past  experience.  Judgment,  and  Intuition  must  be  used  In 
making  the  choice.  For  example,  a  military  headquarters  could  use 
Its  collective  experience  and  Judgment  to  select  values  of  C  and 
Ry  -  Rj^  which  were  felt  to  be  appropriate  when  r^  *  .5.  These 
chosen  values  could  -be  used  to  determine  the  number  of  missiles  to  be 
tested,  by  means  of  confidence-limit  curves  of  the  kind  shown  In  Fig. 

that  were  chosen  were  appropriate 
for  the  headquarters'  needs,  then  this  would  be  a  valid  and  satis¬ 
factory  method  of  choosing  the  number  to  test. 

(4)  Difficulties  In  Making  Those  Choices 

However,  because  the  Implications  of  a  given  choice  are 
not  obvious,  there  will  surely  be  disagreement  as  to  the  most  reason¬ 
able  values.  This  problem  Is  aggravated  by  the  fact  that  we  would 
like  to  employ  the  Judgment  of  military  officers  who  may  not  have 
statistical  training,  and  who  will  find  It  difficult  to  direct  their 
Judgment  towards  the  choice  of  C  and  Ry  -  Rj^.  It  seems  that  a  quantl 
tatlve  assessment  of  the  various  choices  would  have  to  employ  some  or 
all  of  the  additional  Information  used  In  statistical  decision  theory 
and  employ  it  In  rather  similar  ways. 


1.  If  the  values  of  C  and  Ry 
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(3)  Using  the  Classical  Estimates 

We  should  also  notice  here  that  the  second  question, 

"How  to  use  the  results  of  the  test  program  as  a  guide  In  targeting” 

Is  not  readily  answered  by  classical  theory,  even  after  the  experi¬ 
menter  has  determined  the  number  of  missiles  to  test.  The  straight¬ 
forward,  uncritical  use  of  an  unbiased  point  estimate  of  the  reli¬ 
ability  as  If  It  were  the  true  reliability  may  lead  to  poor  decisions, 
particularly  If  the  coat  function  is  biased.  By  using  Judiciously 
biased  point  estimates  one  may  be  able  to  compensate  for  biased  cost 
functions,  but  classical  theory  offers  little  guidance  on  how  to  be 
Judicious  In  this  selection. 

b.  The  Statistical  Decision  Theory  Approach 

We  shall  consider  now  the  statistical  decision  theory 
approach.  Since  the  decision  theory  model  explicitly  Includes  more 
than  the  classical  model,  we  must  gather  more  information  before  the 
model  can  be  used. 

(1)  The  Set  of  Possible  Actions 

For  example,  we  must  formulate  the  purpose  of  the  testing 
in  such  a  way  that  It  refers  to  the  actions  that  may  be  taken  and  to 
the  choice  between  them.  Thus  we  cannot  simply  state  that  we  wish 
to  "discover  the  reliability";  a  suitable  statement  of  the  purpose 
of  the  testing  would  be  "we  test  to  determine  how  to  allocate  missiles 
to  a  target  system."  Thus  the  end  result  of  our  tests  will  be  a 
choice  of  one  allocation  policy.  In  our  example,  we  are  attempting 
to  choose  the  number  of  missiles  to  be  allocated  to  each  target  actu¬ 
ally  attacked  by  these  missiles,  where  we  have  previously  decided  to 
allocate  the  same  nxanber  -  at  most  six  -  to  each  target.  Then  our 
six  possible  policies  are:  allocate  1  missile  to  each  target;  allo¬ 
cate  2  missiles  to  each  target}...}  allocate  6  missiles  to  each  tar¬ 
get.  (Note  again  that  we  are  Ignoring  what  the  missile  engineer  may 
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think  of  as  the  primary  purpose  of  these  tests,  namely,  to  find 
hidden  bugs  In  the  system.  In  fact,  of  course,  operational  tests 
are  never  entirely  separate  from  the  development  testa  —  the  purposes 
overlap  a  little  --  but  the  present  example  restricts  our  attention 
to  the  operational-testing  aspects.) 

(2)  Measure  of  Merit 

The  next  step  is  to  choose  a  measure  of  merit  — 
a  "payof f-functlon"  --  and  to  calculate  its  value  for  each  allo¬ 
cation  and  each  "state  of  nature".*  In  many  practical  problems 
this  choice  Is  the  most  difficult  step  —  one  which  requires  and 
deserves  a  great  deal  of  careful  thought.  The  techniques  of  the 
analysis  are  Independent  of  the  validity  of  the  choice  of  the  pay¬ 
off  function;  of  course,  the  usefulness  of  the  result  depends  cri¬ 
tically  on  this  choice. 

For  our  example  we  have  a  chosen  a  measure  of  merit  based 
on  the  idea  that  we  wish  to  have  a  high  assurance  that  those  targets 
actually  attacked  are  destroyed.  We  choose,  therefore,  the  assurance 
level  that  we  desire,  and  denote  it  by  Q.  Notice  that  this  assurance 
level  Q,  unlike  the  confidence  coefficient  C,  has  an  explicit  opera¬ 
tional  significance .  We  give  ourselves  credit  for  a  target  destroyed 
only  If  the  number  of  missiles  assigned  to  the  target  is  large  enough 
to  make  the  probability  of  destruction  of  the  target  at  least  Q. 

For  this  example  we  make  these  simplifying  assumptions; 
first,  that  the  weapon  yield  and  the  missile  accuracy  are  such  that  a 
missile  that  operates  properly  (l.e.,  a  "reliable"  missile)  will  reach 
the  target  and  destroy  it;  second,  that  all  targets  are  of  equal  value 
and  we  wish  to  achieve  the  same  assurance  of  success  Q  for  each  one; 
third,  that  no  other  weapons  are  used  on  these  same  targets;  fourth, 

*A  '‘state  of  natxire"  Is  simply  a  fact  about  the  real  world,  which  may 
be  considered  as  a  "choice"  made  by  "nature".  The  concept  is  useful 
when  we  are  dealing  with  an  unknown  quantity  (such  as  the  reliability 
In  our  example);  a  value,  or  range  of  values  of  that  quantity  may  be 
called  a  "state  of  nature". 


9 


that  a  missile  will  not  damage  targets  that  It  was  not  aimed  at} 
and  fifth,  that  all  missiles  have  the  same  probability  of  operating 
properly  (this  probability  Is  denoted  by  R  and  Is  called  reliability). 

The  probability  that  a  target  will  survive  an  attack 
by  k  missiles,  each  of  reliability  R,  is  then 

(1  -  R)'", 

and  the  probability  that  It  Is  hit  by  at  least  one  of  those  k 

missiles  Is  . 

1  -  (1  -  R)  . 

Now,  If  we  knew  R  we  could  simply  calculate  the  smallest  k  for 

which  . 

1  -  (1  -  R)  >  Q, 

and  assign  k  missiles  to  each  target. 

It  Is  clear  that  If  testing  cost  nothing,  we  could  test 
until  (with  any  desired  probability  of  any  desired  precision)  we  knew 
the  actual  value  of  R,  and  so  could  choose  the  correct  value  of  k 
for  the  actual  situation. 

We  will  now  show  how  statistical  decision  theory  helps 
us  take  account  of  the  fact  that  testing  Is  not  free.  First  we  draw 
up  a  table  showing  the  payoff  —  the  average  number  of  targets  killed 
per  missile  In  the  stockpile  --  under  various  conditions.  This  pay¬ 
off  Is  of  coxiTse  a  function  of  the  number  of  missiles  required  per 
target  (which  depends  on  the  "state  of  nature",  or  In  this  case  on 
the  value  of  the  unknown  R) ,  and  Is  also  a  function  of  the  number  of 
missiles  actually  assigned  per  target.  For  example,  assume  that  we 
want  90$^  assurance  of  target  destruction  (or,  in  our  notation, 

Q  »  0.90)}  then,  using  the  condition  that  1  -  (l  -  R)*^  must  be  at 
least  Q,  we  can  calculate  the  range  of  R-values  for  which  1  missile 
la  required,  2  missiles  are  required,  etc.,  and  so  generate  the  first 
two  columns  of  Table  1. 
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The  lowest  value  of  R  In  Table  1  la  0.319  because  we 
assume  for  this  example  that  the  results  of  the  R  and  D  test  firings 
permit  us  to  Ignore  the  possibility  of  R  being  less  than  0.319* 

The  zeroes  In  the  lower  left  corner  of  the  payoff 
matrix  result  from  the  assumption  that  no  credit  is  given  for  attack¬ 
ing  a  target  with  fewer  missiles  than  are  required  for  a  probability 
Q  of  destroying  the  target.  The  Commander  wants  to  know  with  905^ 
assurance  that  certain  specific  targets  were  killed.  To  Illustrate 
how  the  rest  of  the  table  Is  constructed  we  calculate  the  starred 
value  In  row  2,  column  4.  We  are  allocating  four  missiles  to  each 
target  so  that  each  four  missiles  suffice  for  one  target.  The 
average  is  1/4  target  per  missile,  and  that  number  of  targets  will 
In  fact  be  killed  with  probability  greater  than  90%,  since  two 
missiles  would  have  been  enough. 

The  reader  may  have  misgivings  about  using  this  particu 
lar  payoff  function.  We  think  that  It  Is  reasonable;  but  In  any  case 
It  will  serve  as  an  example  of  a  biased  payoff  function,  since  allo¬ 
cating  too  few  missiles  Is  much  more  costly  than  allocating  too  many. 

(3)  Prior  Distribution 

Statistical  decision  theory  requires.  In  addition  to 
the  payoff  matrix,  a  "prior  probability  distribution  on  the  states 
of  nature."  This  must  be  developed  as  the  consensus  of  opinion  of 
the  statistician  and  the  executive  and  should  be  an  attempt  to  ex¬ 
ploit  whatever  Information  and  Judgment  are  at  hand.  In  our  hypo¬ 
thetical  missile-testing  example,  we  might  feel  fairly  certain  that 
the  reliability  R  lay  in  a  certain  range j  if  we  could  then  agree  on 
an  estimate  of  betting  odds  for  R  lying  in  various  Intervals,  and 
if  we  could  agree  on  the  general  form  of  the  distribution  of  R,  we 
could  make  an  estimate  of  the  probability  of  each  interval  in  the 
first  column  of  Table  1,  and  could  compute  from  those  numbers  the 
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Calculated  In  the  text  as  an 


probability  of  requiring  various  numbers  of  missiles.  That  estimated 
distribution,  based  on  prior  experience,  is  called  the  prior  (or  £ 
priori)  distribution. 

The  concept  of  prior  probability  distribution  lies  at  the 
center  of  statistical  decision  theory;  it  is  treated  at  length  in 
sections  5  and  6  of  this  paper. 

(4)  Preposterior  Analysis 

Before  we  perform  any  experiment,  or  even  decide  what 
expeiment  to  perform,  we  will  make  what  Raiffa  and  Schlaifer  (ref.  4), 
call  the  preposterior  analysis .  The  purpose  of  that  analysis  is  to 
evaluate  possible  experiments.  In  the  course  of  the  analysis  we  shall 
learn  how  to  use  the  results  of  the  experiments. 

Consider  for  the  moment  some  proposed  experiment  E,  and 
some  one  possible  result  X  of  that  experiment.  (In  our  example, 
choosing  the  experiment  consists  simply  of  choosing  the  number  ^ 
of  missiles  to  be  tested,  and  the  result  can  be  expressed  by  giving 
the  number  m  of  successful  firings  out  of  those  ri  tests.)  On  the 
basis  of  that  result  X  we  will  modify  our  prior  distribution  in  the 
following  way:  each  state  of  nature  that  would  have  made  the  observed 
results  probable  will  be  considered  more  likely  than  before  (i.e., 
its  posterior  probability  will  be  larger  than  its  prior  probability);, 
and  each  state  of  nature  that  would  have  made  the  observed  results 
Improbable  will  be  considered  less  likely  than  before.  (In  particular, 
any  state  of  nature  incompatible  with  the  observed  result  X  will  be 
excluded  from  further  consideration.) 

(a)  Bayes'  Formula 

The  above  ideas  are  made  precise  by  Bayes'  formula,  , 
which,  in  its  general  form  is 

P(AjX)  -  PLxJ.Al.  PiAj  ^ 
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where  P(A)  Is  the  prior  probability  of  the  state  of  nature  A, 

P(X)  is  the  prior  probability  of  observing  the  result  X,  and 
P(a|x)  la  the  posterior  probability  of  the  state  of  nature  A  if 
the  experimental  result  X  has  been  observed.*  The  formula  can  be 
interpreted  in  the  following  way; 

(I)  the  prior  probability  P(A)  of  the  state  of 
nature  A  is  multiplied  by  the  probability  P(x|a)  of  the  observed 
result  occurring  if  A  were  true;  the  resulting  number  P(A).P(x|a) 
measures  the  relative  weight  to  be  given  to  A  in  the  posterior 
distribution j 

(II)  that  set  of  relative  weights  (one  for  each  state 
of  nature  A)  must  be  divided  by  their  sum,  to  get  a  set  of  weight¬ 
ing-factors  (which  must  add  up  to  unity)  ; 

(ill) the  formula  quoted  above  results  as  soon  as  we 
remark  that  the  sum  of  those  numbers,  for  all  possible  A  and  some 
fixed  X,  is  Just  P(X) ;  l.e., 

X^^(p(x|a).p(a))  =  P(X).  ** 

In  our  missile-testing  example,  the  state  of  nature 
corresponds  to  some  value  or  set  of  values  of  the  unknown  reliability 
R,  and  the  test  result  will  be  ro,  the  number  of  missiles  (out  of 
the  total  of  n  tested)  which  were  successful.  Bayes'  formula  now 
lets  us  calculate  P(a|x),  the  posterior  probability  of  the  state  of 
natxire  A  after  we  have  performed  the  experiment  E  and  observed  the 
test  result  X. 

♦The  formula  can  be  established  by  observing  that  the  probability 
of  the  Joint  event  A  and  X  is  expressible  either  as  P(A).P(X|a)  or 
as  P(X).P(a|X))  on  equating  these  expressions  and  dividing  by  P(X), 
Bayes'  formula  results. 

♦♦'The  prior  probability  P(X)  is  the  average  (over  all  states  of 
nature)  of  the  probability  P(X|A)  of  observing  X  if  A  is  true, 
weighted  according  to  the  prior  distribution  P(A). 
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(b)  Optimal  Terminal  Actions 

The  posterior  distribution  obtained  above  represents 
our  Judgment  (as  modified  by  the  test  results)  of  the  plausibility  of 
the  various  states  of  nature.  For  any  one  terminal  action  (l.e.,  in 
our  example,  for  any  one  allocation  policy),  we  have  a  set  of  payoffs, 
one  for  each  state  of  nature;  from  that  set  of  payoffs  we  can  calcu¬ 
late  the  expected  payoff  (using  the  posterior  probabilities  on  the 
states  of  nature  as  weights).  Then  we  may  select,  for  the  proposed 
experiment  E  and  possible  result  X,  that  terminal  action  which  offers 
the  greatest  expected  payoff,  and  record  it  as  the  optimal  terminal 
action.  We  also  record  the  expected  value  of  the  corresponding  pay¬ 
off. 

(c)  Expected  payoff  from  an  experiment 

Consider  now  the  other  possible  outcomes  of  the  partlci 
lar  experiment  E,  and  calculate  for  each  of  them  the  optimal  terminal 
action  and  the  expected  value  of  the  corresponding  payoff. 

When  that  process  has  been  completed,  we  must  con- 
solldPte  those  expected  payoffs,  which  depend  on  the  result  X  as 
well  as  on  E.  For  this  purpose  we  average  them,  using  the  prior  pro¬ 
babilities  P(X)  of  the  various  outcomes  as  weights,  to  obtain  the  over¬ 
all  expected  payoff  of  the  experiment  E. 

(d)  Comparison  of  experiments 

Now  consider  the  totality  of  experiments  under  con¬ 
sideration,  and  perform  the  above  expected-payoff  calculation  for 
each  of  them.  Assess  also  the  costs  of  each  experiment.  If  the 
coats  of  the  experiment  can  be  expressed  in  the  same  terms  as  the 
payoffs,  then  the  "design  of  the  experiment"  consists  only  in 
choosing  that  experiment  which  maximizes  the  difference  (payoff 
minus  cost) .  (The  word  "only"  may  be  misleading,  because  an  enor¬ 
mous  amount  of  calculation  may  be  required  in  this  preposterior 
analysis.)  If,  on  the  other  hand,  the  payoffs  are  incommensurable 
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TABLE  2 
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0.791  X  0.428  +  0.209  x  0.510  »  0.404  targets  per  Blaalle 


with  the  costs  of  the  experiment  (e.g.,  the  cost  of  an  experiment  Is 
dollars,  while  the  payoff  is  military  effectiveness  of  one  kind  or 
another)  it  is  necessary  for  the  statistician  to  confer  with  the 
decision-maker,  and  employ  the  latter's  Judgment  about  the  operation¬ 
al  implication  of  the  various  payoffs,  to  decide  which  experiment  to 
perform.* 

Note  that  we  have  already  made  the  calculations 
which  tell  us  which  terminal  action  to  take  when  the  experiment  la 
complete  and  its  result  has  been  observed. 

(5)  Computations  for  the  example. 

In  Table  2,  a  possible  prior  distribution  (whose  origin 
will  be  described  in  section  6  below)  has  been  Inserted  as  column 
4.  Table  2  represents  the  same  situation  as  Table  1,  with  two  changes: 
(a)  the  range  of  R,  within  which  two  missiles  per  target  are  required, 
has  been  divided  into  two  smaller  ranges  in  order  to  Improve  the  pre¬ 
cision  with  which  later  calculations  can  be  made;  (b)  the  midpoints 
of  the  ranges  of  R  have  been  Introduced  in  order  to  have  a  point  value 
for  use  in  later  calculations. 

We  consider  first  the  experiment  n  =  1,  in  which  we  test 
one  missile.  (Throughout  this  example  we  assume  that  the  test  missiles 
do  not  come  from  the  stockpile,  so  that  the  number  of  missiles  avail¬ 
able  to  fire  against  targets  is  not  reduced  by  the  test  program.)  The 
possible  outcomes  are  m  =  0  (no  successes,  l.e.,  the  missile  falls) 


*The  concept  of  cost  associated  with  testing  may  be  a  much  richer  con¬ 
cept  than  the  term  "cost"  ordinarily  suggests.  Statistical  decision 
theory  permits  it  to  Include  not  only  the  dollar  cost  of  production 
of  test  missiles,  but  also,  for  example,  as  negative  costs,  the  advan¬ 
tages  accruing  from  keeping  production  lines  operating  and  from  the 
crew  training  and  experience  which  result  from  test  firing.  Insofar 
as  this  enrichment  adds  Incommensurable  elements  to  the  cost,  making 
cost  a  vector  instead  of  a  scalar,  it  complicates  the  later  choice  of 
an  experiment!  but  the  principles  of  the  statistical  theory  are  not 
affected. 
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TABLE  3 

Calculation  of  F(X) 


State  .of 
Natxire 

A 

Prior 

Probability^ 

P(A) 

p(x|a) 
Conditional 
Probability 
of  Success 

p(x|a).p(a) 

1 

.210 

.950 

.199 

2 

.322 

.850 

.274 

3 

.276 

.742 

.205 

4 

.161 

.610 

.098 

5 

.024 

.487 

.012 

6 

.005 

.404 

.002 

7 

.002 

.344 

.001 

Total:  .791  =  P(X). 
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and  m  <■  1  (one  success^  l.e.«  the  missile  succeeds.)  1%e  posterior 
probabilities  for  m  »  0  or  1  respectively  are  given  In  the  two 
columns  at  the  extreme  right  of  Table  2,  which  are  calculated 
directly  from  Bayes*  formula  (cf,  paragraph  4b(4)(a)).  For  exasple^ 
let  us  calculate  the  second  entry  In  the  last  coliunn.  Bayes*  formula 

13  P(A|X)  .  ,  Where  the  A  Is  state  of  nature  No.  2,  0.800 

<  R  5  0.899;  X  Is  the  result  "the  missile  succeeds";  P(A)  =  0.522  from 
column  4  of  Table  2;  P(X|A)  Is  the  probability  of  X  (l.e.,  success) 
given  that  R  Is  in  the  specified  range,  and  we  take  O.85O  as  a  mld> 
range  representative  value  of  the  reliability,  so  that  P(X|A)  =  O.85O. 
Thus  P(x|a).P(A)  =  0.850  X  0.322  =  0.274.  To  calculate  P(X)  we  must 
do  the  same  calculations  for  all  values  of  A,  and  form  the  sum  as 
shown  in  Table  5.  We  find  that  P(X)  =5~!^P(X| A) . P( A)  =  0.791.  Then 

P(a|x)  =  —  "  0.546,  as  appears  in  the  second 

place  of  the  last  column.  Table  2. 

The  expected  payoff  (in  targets  per  missile  in  stockpile), 
for  each  of  the  six  methods  of  missile  assignment  considered,  is 
shown  at  the  foot  of  the  column  corresponding  to  that  assignment  In 
Table  2:  each  value  in  the  row  called  "Expected  Payoff,  Test  1, 

Fall"  is  obtained  by  averaging  the  seven  payoffs  in  the  corresponding 
column,  using  the  weighting- factors  given  by  the  posterior  probabili¬ 
ties  for  m  =  0;  the  values  in  the  row  called  "Expected  Payoff,  Test  1, 
Succeed"  are  similarly  calculated  using  the  posterior  probabilities 
for  m  =  1. 

The  maximum  value  In  each  row  Is  underlined;  It  happens 
to  be  the  value  corresponding  to  "allocate  two  missiles  per  target," 
both  for  the  "Pall"  row  and  for  the  "Succeed"  row.  The  decision- 
rule  Inqplled  by  these  calculations,  if  we  decide  to  test  one  missile. 
Is  to  allocate  two  missiles  per  target,  whether  the  missile  succeeds 
or  not.  The  overall  expected  payoff.  In  case  we  decide  to  test  one 
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TABLE  A 


missile.  Is  to  allocate  two  missiles  per  target,  whether  the  missile 
succeeds  or  not.  The  overall  expected  payoff.  In  case  we  decide  to 
test  one  missile.  Is  equal  to  the  probability  of  failure  tljoes  the 
payoff  expected  In  case  of  failure,  plus  the  probability  of  success 
times  the  payoff  expected  In  case  of  success.  This  Is  the  calculation 
at  the  bottom  of  Table  2j  since,  as  shown  In  Table  3#  the  overall  pro¬ 
bability  of  success  Is  0.791#  the  overall  probability  of  failure  must 
be  0.209,  and  the  expected  payoff  la  0.404  targets  per  missile. 

Naturally,  we  include  In  the  set  of  "experiments” 
to  be  considered  the  case  where  we  do  no  testing.  This  Is  the  case 
n  =  0  (test  no  missiles),  and  the  posterior  distribution  la  of  course 
Identical  with  the  prior  distribution.  The  expected  payoff  for  each 
method  of  allocation.  If  we  take  n  =  0,  la  given  In  the  row  of  Table 
2  labeled  "Expected  Payoff,  No  Testing."  Each  of  these  values  was  ob¬ 
tained  by  averaging  the  payoffs  In  the  corresponding  column,  using 
In  this  case  the  prior  probabilities  as  weights.  Again,  the  maximum 
value  would  have  been  obtained  when  two  missiles  were  allocated  per 
target,  and  the  expected  payoff  would  have  been  0.4o4.  We  see  that 
testing  one  missile  would  not  Increase  the  expected  number  of  targets 
killed  per  missile.  This  is  not  surprising  when  we  realize  that  we 
would  take  the  same  action  (allocate  two  missiles)  for  each  of  the  two 
possible  outcomes  of  the  single  test;  since  no  possible  outcome  of  the 
experiment  would  affect  our  action,  the  test  is  not  worth  doing.  (Of 
course,  such  a  test  might  affect  other  actions  not  considered  In  the 
model,  and  It  might  be  worth  doing  for  those  other  reasons.) 

We  may  make  similar  calculations  to  decide  whether 
we  wish  to  test  more  than  one  missile.  For  example.  Table  4  shows 
the  calculations  for  the  proposal  to  take  n  »  2,  (l.e.,  to  test  two 
missiles.)  The  expected  payoff  Is  then  0.408  targets  per  missile  In¬ 
stead  of  0.4o4.  If  20  missiles  are  tested  (the  calculations  are  not 
exhibited  in  this  paper)  the  expected  payoff  Is  0.480  targets  per 
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stockpile  missile.  If  40  are  tested,  the  expected  payoff  Is  0.3X2 
targets  per  missile.  (For  example,  with  a  stockpile  of  300  mlaslXes, 
we  find  that  testing  two  missiles  increases  the  number  of  targets 
killed  from  ]21  to  122,  testing  20  missiles  Increas  s  the  number  of 
targets  killed  to  144,  and  testing  40  Increases  It  to  15^»)  This 
Information  is  summarized  In  Figure  3,  page  34. 

The  results  of  calculations  of  this  type,  as  pre¬ 
sented  In  Figure  3,  furnish  the  data  needed  to  make  a  decision  on 
how  many  missiles  to  test.  Hewever,  the  use  of  data  of  this  kind  to 
decide  how  many  missiles  to  test  may  require  military,  political,  and 
economic  Judgment.  The  statistician,  as  such,  is  not  qualified  to 
make  the  decision;  although  his  advice  should  be  given  when  asked  for, 
the  final  decision  should  not  be  his. 

(6)  Employment  of  the  Computational  Results 

Our  exposition  so  far  has  shown  how  information  from 
missile  testing  can  be  used  to  increase  the  number  of  targets  des¬ 
troyed  by  a  given  missile  force.  The  choice  of  the  number  of  missiles 
to  test  would  appear  to  require  simply  a  comparison  of  the  cost  of 
testing  and  the  value  of  the  extra  targets  destroyed.  But  a  direct 
comparison  of  these  two  variables  can  only  be  made  when  they  are  both 
measurable  in  a  common  unit  (such  as  dollars).  We  first  give  In 
paragraph  4.b.(6)(a)  an  example  showing  how  the  choice  Is  made  when 
such  a  common  unit  does  not  exist;  then  the  example  of  paragraph 
4.b.(6)(b)  Illustrates  how  to  use  such  a  common  unit,  in  case  it  does 
exist. 

(a)  In  the  present  section  we  show  how  the  choice  may  be 
made  without  an  explicitly  stated  common  unit  of  measurement  for  cost 
of  missiles  tested  and  value  of  targets  destroyed. 

The  essence  of  the  method  Is  a  liberal  application 
of  Intuition  and  Judgment  to  numbers  of  the  kind  presented  In  Table  5$ 
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which  are  prepared  by  the  atatlatlclan.  The  intuition  and  ^udgiMnt 
are  furnished  by  the  executive,  who  examines  the  rows  that  correspond 
to  possible  stockpiles  —  Just  one  row  If  he  knows  what  the  stockpile 
Is  or  will  be.  Let  us  assume  that  the  stockpile  Is  firmly  established 
at  300  missiles.  The  executive  examines  the  number  of  targets  killed 
In  the  300-mlsslle  stockpile  row,  noticing,  for  example,  that  the 
first  10  missiles  tested  buy  12  extra  targets  destroyed  (from  121  to 
133)  and  that  ten  missiles  tested  after  40  have  already  been  tested 
buy  only  three  extra  targets  destroyed  (from  154  to  157). 

In  some  tactical  situations  an  extra  three  targets  des¬ 
troyed  may  be  very  Important.  If  the  executive  predicts  that  this 
Is  the  situation  he  will  be  faced  with,  he  may  decide  to  te.st  50 


missiles.  On 

the  other 

hand. 

he  may 

feel  that 

the  150 

targets  killed 

TABLE 

No.  of  Ms  Is 
tested 

.  0 

10 

20 

_ 30 _ 

40 

50 _ 

Msl 

Stockpile 

100 

40 

45 

48 

50 

51 

52 

200 

81 

89 

95 

100 

102 

104 

300 

121 

133 

144 

150 

154 

157 

400 

162 

178 

191 

200 

205 

209 

500 

202 

223 

239 

250 

■HHi 

261 

23 

i 

S' 

I 


with  30  mlBBiles  teBted  1b  the  right  place  to  operate*  Bince  the 
extra  four  targets  killed  by  Increasing  from  30  to  40  mlSBlles  tested 
are  not  sufficiently  In^ortant  to  Justify  the  cost  of  testing  10 
mlBBlles.  If  the  actual  stockpile  Is  not  firmly  established* 
possible  stockpiles  are  examined  Just  as  the  300  stockpile  was 
examined;  the  choice  of  number  of  missiles  to  test  Is  made  by  sub¬ 
jective  weighting  of  the  choices  for  each  stockpile. 

Of  course,  analysis  of  this  kind  Implicitly  involves 
a  common  unit  for  measuring  cost  of  missiles  tested  and  value  of  tar¬ 
gets  destroyed.  Nevertheless,  a  rational  executive  may  find  this 
method  of  decision  more  congenial  than  one  which  requires  him  to  be¬ 
gin  by  making  an  explicit  assignment  of  a  dollar  value  to  targets  des¬ 
troyed. 

(b)  For  this  second  example,  we  assume  that  the  cost 
and  value  data  can  be  expressed  In  a  common  unit.  Specifically,  we 
assume  that  the  cost  of  testing  n  missiles  Is  simply  nC  dollars, 
where  C  Is  the  cost  of  testing  one  missile,  and  that  the  value  of 
attacking  one  additional  target  can  be  expressed  in  monetary  terms 
as  KC  dollars,  where  K  Is  some  factor  (presumably  greater  than  l) 
which  we  must  estimate.  For  example,  If  the  value  of  attacking  one 
additional  target  was  considered  equal  to  the  cost  of  a  test  missile* 
we  would  assign  the  value  of  K  »  1;  If  the  value  of  attacking  an 
additional  target  was  considered  to  be  ten  times  the  cost  of  a  test 
missile,  we  would  assign  the  value  of  K  «  lo.  Believing  that  a 
target  attacked  Is  worth  much  more  to  us  than  the  cost  of  the  missile 
attacking  It,  we  may  feel  that  K  should  be  between  1  and  10. 

Denote  by  T(n)  the  average  number  of  targets  struck 
per  missile*  If  we  test  n  and  make  the  optimal  allocation;  a  graph 
of  T(n)  against  n  Is  given  In  Figure  3.  Then  the  total  number  of 
targets  attacked*  If  we  have  a  stockpile  of  S  missiles  and  test  n 
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mlBBlles,  iB  ST(n)«  and  the  value  to  ub  of  attacking  these  targetB, 
at  KC  dollars  per  target.  Is  KCST(n).  TItsB  cost  of  testing  n 
missiles  is  nC,  so  we  wish  to  maximize  the  difference  KCST(n)  -  nC, 
which  equals  KCS(T(n)  -  )•  This  will  be  maximized  wl»n  the  curve 

T(n),  plotted  as  a  function  of  n  as  In  Figure  5,  (page  34),  has 
slope  (KS)”^.  Tangent  lines  are  drawn  on  Figure  5  with  slopes 
1  ,  and  1  . showing  that: 

ms  ^  im 

for  KS  =  100  --  either  K  *  1  and  a  stockpile  of 
100  missiles,  or  K  »  10  and  a  stockpile  of  10 
missiles  —  we  should  do  no  testing; 

for  KS  *  254  it  is  a  matter  of  indifference 
whether  or  not  we  do  any  testing,  but  if  we  do 
test  we  should  test  about  11  missiles;  and 

for  KS  =  1000,  we  should  test  about  44  missiles. 

If  we  take  K  =  3  for  example,  we  show  in  Table  6 
how  the  nu.nber  tested  depends  on  the  stockpile  S.  In  this  case  a 
stockpile  of  300  missiles  would  imply  that  we  should  test  40  missiles. 

TABLE  6 


- - 

n. 

Number  of  Missiles 

Number  of 

Missiles 

in  Operational  Stockpile 

to 

be 

tested 

<  78 

0 

78 

0 

or 

11 

100 

15 

120 

18 

150 

22 

200 

29 

300 

40 
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5.  Another  Objection 

The  key  to  the  procedure  that  we  have  expounded  above  Is  the 
assignment  of  a  prior  distribution  to  the  states  of  nature. 

a.  The  strongest  argument  against  the  procedure  Is  that  the 
concept  of  a  prior  distribution  of  a  parameter  (in  our  example,  R) 

Is  logically  meaningless;  It  can  be  argued  that  R  Is  not  a  random 
variable,  but  a  fixed  (though  unknown)  value.  This  question,  which 
really  concerns  the  foundations  and  philosophy  of  probability  theory, 
has  not  received,  and  probably  never  will  receive,  universally 
accepted  answers.  Nevertheless,  some  counter- arguments  can  be  made: 

(1)  First,  for  classical  statistics  to  be  useful  at  all,  one 
must  in  some  way  weight  the  possible  states  of  nature.  B'urthermore , 
the  Implicit  weighting  which  characterizes  most  applications  of 
classical  statistics  Is  subject  to  the  same  logical  objections  as 
Bayesian  statistics,  but  fails  to  yield  any  of  the  benefits  of  the 
Bayesian  methods.  For  example,  one  decides  about  the  alternatives 
among  which  one  is  going  to  choose,  and  one  thereby  completely 
excludes  from  formal  consideration  a  host  of  Imaginable  alternatives. 
As  another  example.  In  setting  up  an  unbiased  estimator  as  a  desirable 
type  (as  Is  normally  done  In  classical  applications)  one  Is  Implicitly 
assuming  that  the  expected  cost  of  an  error  In  one  direction  Is  equal 
to  the  expected  cost  of  an  error  In  the  other  direction  —  which  can 
only  be  true  for  one  ratio  of  the  relative  probabilities.  Calling 
the  prior  distribution  on  the  states  of  nature  a  weighting-function 
(which  might  be  subjective),  rather  than  a  probability  distribution. 

Is  permissible  (since  both  satisfy  the  postulates  of  probability 
theory)  If  that  soothes  the  conscience  of  the  logical  objector. 

(2)  Second,  any  statement  of  probability  can  be  regarded  as 
an  expression  of  Ignorance.  Before  we  flip  a  coin,  most  people  will 
permit  us  to  say  that  the  probability  of  a  head  is  1/2  (or  perhaps 
some  other  number  near  1/2).  After  the  coin  has  been  flipped,  but 


26 


before  we  have  seen  it,  the  probability  of  a  head  Is  surely  either 
1  (if  It  Is  a  head)  or  0  (if  It  Is  a  tall).  However^  It  may  not  be 
wholly  unreasonable  to  maintain  that  the  (subjective)  probability, 
until  we  obtain  some  Information  about  the  result,  should  be  con¬ 
sidered  1/2.  Setting  a  prior  distribution  on  a  fact  (namely,  the 
actual  but  unknown  value  of  R)  seems  to  be  similar  In  spirit. 

(3)  Third,  modern  physics  forces  us  to  admit  the  possibility 
that  length,  position,  velocity,  etc.,  are  primarily  probability 
distributions,  which  only  superficially  appear  to  be  definite  values, 
rather  than  the  converse.  And  the  reliability  of  a  missile,  or  the 
truth  of  a  hypothesis,  can  hardly  be  considered  more  concrete  and 
deterministic  than  the  position  of  a  particle. 

b.  Another  difficulty,  even  if  one  admits  that  the  concept  of 
a  prior  distribution  can  be  meaningful,  is  that  the  determination 
of  such  a  distribution  may  be  arbitrary,  subjective,  and  unscientific. 
Several  remarks  must  be  made  in  answer: 

(1)  First,  it  may  happen  that  the  prior  distribution  actually 
has  little  or  no  effect  on  the  decisions  we  make.  In  our  missile- 
testing  example,  the  Influence  of  the  prior  distribution  on  our 
allocation  of  missiles  will  be  less  Important  if  large  numbers  of 
missiles  are  tested;  then  prior  distributions  that  appear  very  differ¬ 
ent  may  lead  to  decisions  that  are  very  similar. 

(2)  Second,  the  existence  of  a  power  known  as  "judgment”  shows 
that  subjective  opinions  need  not  be  arbitrary  and  may  be  scientific. 
Such  opinions  may  be  based  on  a  body  of  facts  and  experience  which 
are  valid  even  though  we  cannot  quote  them  explicitly. 

Third,  If  it  Is  true  (as  we  maintain)  that  one  cannot 
draw  any  conclusions  from  an  experiment  without  some  prior  assumptions 
then  the  difficulty  of  making  good  prior  assumptions  Is  regrettable 
but  Irrelevant.  For  exanple,  Occam's  Razor  ("Prefer  the  simplest 
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hypothesis  consistent  with  the  facts”)  can  he  regarded  as  a  prejudice 
in  favor  of  simple  physical  laws;  and  modem  science  could  hardly 
exist  without  that  unscientific  prejudice. 

(4)  Finally,  almost  any  experln»nt  must  he  Inccamplete  in 
the  sense  that  its  subjfect  is  connected  to  other  questions,  on 
which  experiments  might  also  be  performed.  To  make  a  problem 
finite,  we  must  ignore  many  of  those  connections.  However,  the 
posterior  distributions  which  resulted  from  e:qperlments  previously 
performed  may  Influence  (or  even  determine)  the  prior  distribution 
for  the  present  experiment  —  and  the  posterior  distribution  of  the 
present  experiment  may  Influence  the  prior  distribution  for  subse¬ 
quent  experiments.  (In  other  words,  a  parameter  estimated  by  one 
experimenter  should  be  used  by  later  experimenters  with  due  regard 
to  the  variability  of  the  estimate).  If  the  result  of  any  particular 
experiment  is  very  surprising,  we  may  be  forced  to  retract  our  prior 
distribution,  saying,  "This  is  a  paradox”.  Vfhen  the  paradox  has 
been  explained,  a  quite  different  prior  distribution  may  be  postu¬ 
lated  for  the  next  experiment;  if  the  results  of  this  new  experiment 
are  then  within  the  new  expected  range,  we  say,  "A  new  theory  has 
been  confirmed” , 

6.  A  Practical  Problem 

If  we  accept  the  principle  of  a  prior  probability  distribution, 
we  are  left  with  the  practical  problem  of  estimating  its  numerical 
values.  This  is  never  easy,  but  frequently  one  can  choose  the 
general  form  of  the  distribution  from  first  principles,  and  en^loy 
what  knowledge  we  have  to  estimate  the  actual  distribution. 

For  exan5)le,  the  numbers  given  in  column  3  of  Table  2  were  ob¬ 
tained  in  five  steps.  First,  we  assumed  that  a  binomial  distribution 
with  some  (unspecified)  mean  would  give  a  distribution  of  the  right 
general  shape.  Second,  we  assumed  that  R,  the  reliability,  was  not 
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less  than  0.^2  —  l.e.,  we  assumed  that  not  more  than  6  islssiles 
would  be  required  to  give  90$^  assurance  of  a  kill.  Thlrdj»  on  the 
basis  of  hypothetical  research  and  development  tests,  aided  by 
opinions  on  the  effects  of  the  changes  made  during  development, 
we  guessed  that  R  looked  like  0,8  or  so.  Fourth,  a  binomial 
distribution  with  mean  0.8  was  selected  which  gave  a  reasonable 
spread  of  values  for  Rj  we  actually  chose  the  binomial  distribution 
of  10  trials.  Fifth,  we  drew  a  smooth  curve  through  the  points 
representing  that  distribution,  as  shown  In  Figure  2,*  and  drew  verti¬ 
cal  lines  separating  the  ranges  of  R  in  which  1,  2,  6,  missiles 

were  required.  Sixth,  by  integrating  the  various  regions  under  the 
curve,  we  estimated  the  probability  of  R  lying  in  each  of  the  ranges 
considered.  Those  prior  probabilities  were  given  in  column  5  of 
Table  2. 

7 .  Summary 

We  see  that  the  Bayesian  approach  has  done  the  following  for 
us: 

a.  It  has  permitted  a  direct  comparison  of  tangible,  understand¬ 
able  values  VO  decide  whether  testing  pays,  and  to  permit  us  to 
choose  the  number  to  test.  (As  discussed  In  paragraph  4c  and  foot¬ 
note,  the  direct  comparison  may  nevertheless  Involve  a  high  order 

of  Judgment). 

b.  For  whatever  number  we  decide  to  test.  It  tells  us  explicitly 
how  to  use  the  test  results  to  allocate  missiles  to  targets. 

c.  It  permits  an  experiment,  considered  as  a  process  for  gather¬ 
ing  Information,  to  be  regarded  from  a  unified  viewpoint.  Instead 

of  beginning  In  a  vacuum  (with  no  Information  about  the  objects  under 
test)  and  ending  with  a  confidence-interval  (whose  Interpretation 
In  practical  terms  Is  very  difficult),  we  begin  with  opinions  (in 
the  form  of  the  prior  distribution)  as  to  the  relative  likelihood 
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of  the  various  posslhllities,  and  end  up  with  modified  q^lhlcms 
(in  the  form  of  the  posterior  distribution)  on  those  likelihoods. 

This  makes  it  mush  more  reasonable  to  split  up  a  complex  set  of 
experiments  Into  sub -experiments,  using  the  posterior  distribution 
from  one  experiment  as  the  prior  distribution  for  another,  and 
makes  It  possible  to  relate  dlstlnc-.t  experiments  on  the  same  subject 
In  a  consistent  way. 

If  the  possible  number  of  missiles  one  might  test  is 
large,  hand  calculations  of  the  kind  presented  in  our  examples  for 
n  -  0,  1,  and  ?,  would  be  very  arduous.  Two  alternatives  are  avail¬ 
able:  for  some  payoff  functions  and  some  prior  probability  distri¬ 

butions,  closed-form  soluMons  (formulae  for  quickly  calculating 
the  answers)  can  be  obtained;  when  closed-form  solutions  are  not 
possible  the  numerical  results  may  be  generated  on  a  high-speed 
computer.  Research  on  the  for-m.er  attack  is  going  forward  at  Stanford 
University  and  the  University  of  Chicago  under  APCOA  sponsorship. 

In  addition,  a  program  for  the  IBM  7^*90  has  been  written  at  AFCOA 
which  permits  a  wide  choice  of  prior  distributions,  and  which  cal¬ 
culates  the  payoff  in  anccrdanre  with  the  methods  of  this  example. 

A  subsequent  OA  paper  will  describe  the  program  in  detail. 

3.  Conclusion 

It  is  our  opinion  that  the  information  for  applying  statistical 
decision  theory,  including  a  prior  distribution,  can  often  be  esti¬ 
mated  and  made  available.  In  many  practical  cases,  more  operation¬ 
ally  meaningful  results  can  be  obtained  by  using  statistical  decision 
theory  than  by  using  the  classical  statistical  estimation  theory. 
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(Attn:  Prof.  Solomon) .  1 

Systems  Development  Corporation,  2500  Colorado  Avenue, 

Santa  Monica,  California .  . . . . .  1 

University  of  Chicago,  Laboratories  of  Applied  Sciences, 

Museum  of  Science  and  Industry,  Chicago  37,  Illinois .  1 

University  of  Chicago,  Dept  of  Statistics,  Chicago,  Illinois, 

(Attn:  Prof.  Kruskal) . . .  1 

University  of  Georgia,  Athens,  Ga.,  (Attn:  Prof.  Cohen) .  1 
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CONSULTANTS 


No*  of  ■  I 


COPiOB 


Professor  Holt  Ashley,  Massachusetts  Institute  of  Technology 

Cambridge,  Mass . 1 

Dr.  Leroy  A.  Brothers,  Dean,  Dept,  of  Engineering,  Drexel 

Institute  of  Technology,  Philadelphia  4,  Pennsylvania..  1 

Dr.  Wendell  Dwyer,  Westlnghouse,  1000  Connecticut  Ave.,  N.W., 

Washington,  D.C .  1 

Dr.  Roland  Larson,  Applied  Physics  Laboratory,  Johns  Hopkins 

University,  Silver  Spring,  Maryland .  1 

Dr.  George  E.  Nicholson,  Jr.,  Department  of  Statistics, 
University  of  North  Carolina,  Chapel  Hill,  North 
Carolina . 1 

Dr.  Martin  D.  Schwartz,  North  American  Aviation  Corporation, 

Downey,  California .  1 

Professor  Jack  Sllber,  Department  of  Mathematics,  Roosevelt 

College,  Room  4l8,  Chicago  5,  Illinois .  1 

Dr.  W.  Norris  Tuttle,  General  Radio  Corp,,  Concord,  Mass....  1 

Mr,  Roger  Wilkinson,  Bell  Labs,  New  York,  New  York .  1 

OTHER 

U.S.  Operations  Research  Groups,  Liaison  Representative 

U.K.,  Box  92,  USN  100  FPO,  New  York,  N.Y .  1 

Shape  Air  Defense  Technical  Center,  (SADTC),  The  Hague, 

Netherlands .  1 

Assistant  Secretary  General  for  Scientific  Affairs,  NATO, 

Place  du  Marechal  de  Lattre  de  Tasslgny  (Porte 
Dauphlne),  Paris  XVI,  France .  1 

Auburn  University,  Mechanical  Engineering  Dept,  Auburn, 

Alabama  (Attn;  Prof  J.  Grady  Cox) . 1 
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